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Abstract 

In this paper we consider exact tests of a muhiple logistic regression, where the 
levels of covariates are equally spaced, via Markov beses. In usual application of 
multiple logistic regression, the sample size is positive for each combination of levels 
of the covariates. In this case we do not need a whole Markov basis, which guarantees 
connectivity of all fibers. We first give an explicit Markov basis for multiple Poisson 
regression. By the Lawrence lifting of this basis, in the case of bivariate logistic 
regression, we show a simple subset of the Markov basis which connects all fibers 
with a positive sample size for each combination of levels of covariates. 
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O ■ 1 Introduction 



Diaconis and Sturmfelsl 1998 ] developed an algorithm for sampling from conditional dis- 



tributions for a statistical model of discrete exponential families, based on the algebraic 
theory of toric ideals. This algorithm is applied to categorical data analysis through the 
notion of Markov bases. However, often Markov bases are large and difficult to compute. 
One reason for their large size is that they guarantee connectivity of all fibers (conditional 
sample spaces). With a given particular data set, on the other hand, we are naturally in- 
terested in the connectivity of a particular fiber. However obtaining a subset of a Markov 



basis for connecting a particular fiber is also a difficult p roblem in general [Chen et al.L 



2008 1 . This problem was alrea dy discussed in Section 3 o f Diaconis and Sturmfelsl 19981 



concerning "corner minors" . In lAoki and Takemural [2005l | the case of two-way incomplete 
tables was studied. 
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In most applications of the logistic regression model, for each combination of covari- 
ates, the number of successes and the number of failures are observed. The number of 
trials (i.e. the sum of numbers of successes and failures) for each combination of covariates 
is usually fixed by a sampling scheme and positive. We call this marginal the response 
variable marginal. Therefore we are usually interested in the connectivity of fibers with 
positive response variable marginals rather than all the fibers. First, in this paper, we 
show an expl icit form of a Mark ov basis for multiple Poisson regression. Then, extending 



the result of iChen et al.l j2005l | , we show an explicit form of a subset of Markov basis, 
which guarantees the connectivity of every fiber with positive response variable marginals 
for bivariate logistic regression. We conjecture that a similar subset of Markov basis 
connects fibers with positive response variable marginals for a general multiple logistic 
regression. However, it seems difficult to prove this conjecture. 

The logistic regression can be understood as the Lawrence lifting of a Poisson regres- 
sion. Let A denote a configuration defining a toric ideal and let A{A) de note its Lawrence 



lifting. Let I a and Ia{A) denote the respective toric ideals. It is known [Sturmfelsl . 11996 , 
Theorem 7.1] that the unique minimal Markov basis of Ia(A) coincides with the Graver 
basis of I A- Therefore the whole Graver basis of I a is needed to guarantee the connectivity 
of all fibers of A{A). However many of the elements of the Graver basis of I a seem to be 
needed to cope with the case of zero response variable marginal frequencies. In Section 
m for the case of bivariate logistic regression, we prove that a smaller Markov basis for 
the Poisson regression extended to the logistic regression guarantees the connectivity of 
fibers with positive response variable marginals. 

This paper is organized as follows. In Section [2] we summarize results on Markov basis 
of univariate Poisson regression and results on the connectivity for fibers with positive 
response variable marginals of univariate logistic regression. In Section [3] we prove a 
theorem on Markov bases of Segre product of configurations and apply it to multiple 
Poisson regression. In Section |4] we prove the connectivity of fibers with positive response 
variable marginals in the case of bivariate logistic regression. Some numerical examples 
are given in Section [51 We conclude this paper with some discussions in Section [61 Some 
detailed proofs are in Appendix. 



2 Univariate Poisson and logistic regressions 



In this section we summarize results on Markov basis of univariate Poisson regression and 
the connectivity results for fibers with positive response variable marginals of univariate 
logistic regression. We provide exact statements and detailed proofs of these results, 
because they are not explicitly given in literature and similar arguments will be repeatedly 
applied to prove our main theorem in Secti on [H 



Consider univariate Poisson regression [Diaconis et al.l . Il998| with the set of levels 
{1,..., J} of a covariate. The mean fij of independent Poisson random variables Xj, 
j = 1, . . . , J, is modeled as 



log fij = a + (3j, j = l,...,J. 
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The sufficient statistic for the models is Xj, J^j)- The ffist component is the 

total sample size n = The configuration A, i.e. the matrix giving the relation 

between the observation vector and the sufficient statistic, for this model is given by 



A 



1 1 
1 2 



Now we show the minimum-fiber Markov basis [Takemura and Aokil . l2005l | for the uni- 
variate Poisson regression. The minimum-fiber Markov basis is the union of all minimal 
Markov bases. 

Proposition 1. Let ej denote the contingency table with just 1 frequency in the j-th cell. 

The set of moves 



^ = {±(eii + - - ej.^) | 1 < ji < j2 < h < k < J, h - Ji = J4 - Ja} (2) 
forms the minimum-fiber Markov basis for the univariate Poisson regression. 



Proof. We employ the distance reducing argument of iTakemura and Aokil 20051]. Let 



X = {xi, . . . , xj) and y = [yi, . . . , yj), x y, he two data sets in the same fiber. Let 
ji = min{A; | Xk yk}- Consider the case Xj^ > yj-^. The case Xj-^ < yj^ can be handled by 
interchanging the roles of x and y. Because the total sample size n is the same in x and 



y (i.e. n = ^i^iXk = J2k=iyk)y there exists some j2 > ji such that xj^ < Vj^- Choose 



the smallest such j2- Now suppose that Xk < Vk for all k > j2. Then 

= > k{yk - Xfc) > ^ j2{yk - Xk) - ^ij2 - l)ixk - yk) 

k=j2 

Xk) + y^jxk 



J 

E 

k=l 



J 

j2^{yk 

k=l 



k=l 
j2-l 



Vk) 



k=l 



k=l 



Xk - Vk, 



> 0, 



which is a contradiction. Therefore there exists some > j2, such that xj^ > yj^. Define 



ji+ji-j2- Then e 



'J2 



can be subtracted from x and the Li distance 



J3 

to y becomes smaller. This proves that B forms a Markov basis. 

Now consider a fiber JF2 ^ with sample size n = 2 and a particular value of c 
Ylk=i ^Xk- This fiber is written as 



^2,c = {ej + e; I 1 < j < 3' < J, j+j' = c} 

and B consists of all the differences of two elements of these fibers. Since B forms a Markov 
basis, every minimal Markov basis needs to connect only these fibers. This proves that B 
is the minimum-fiber Markov basis □ 



We now consider univariate logistic regression Chen et al.l . |2005| |. Let {1, . . . , J} be 
the set levels of a covariate and let Xij and X2j, j = 1, . . . , J, be the numbers of successes 
and failures, respectively. The probability for success pj is modeled as 



logit(pj) = log' 



Pj 



« + 3 



J. 
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The sufficient statistics for the model is . . . ,X^j, J2j=ij-^+j)- Hence moves 

z = {zij) for the model satisfy (2:1+, z^i, . . . , z+j) = and 



J 



The configuration for this model is the Lawrence lifting A{A) of A in ([1]): 

where Ej denotes the J x J identity matrix. 

In general Markov bases of A{A) become very complicated. In usual applications of 
the logist ic regressioii mode l, however, X+j := Xij+X2j is fixed by a sampling scheme and 



positive. I Chen et al.l [20051 ] showed that a simple subset of Markov bases of A(A) guar- 
antees the connectivity of all fibers satisfying (X+i, . . . ,X+j) > 0, where the inequality 
"> 0" means that every element is positive. 

Let Cj be redefined by a 2 x J integer array with 1 in the (l,j)-cell and —1 in 
the (2,j)-cell. Then we can show that the set of moves in ([2]) con nects all fibers with 



iX±i, • • • , X+j) > 0. More strongly, the set of moves is norm-reducing jTakemura and Aoki 
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2005| for any two tables x, y in any fiber with positive marginals, i.e. we can make the Li 



distance between x and y smaller by a move from the set. 
Proposition 2. The set of moves 

Ba{a) = {±(eji + ej^ - Sj^ - 6^3) | 1 < ji < j2 < js < J4 < J, 32 - ji = Ja - js} (5) 

is norm-reducing for all fibers with (X+i, . . . , X^j) > for the univariate logistic regres- 
sion model. 

To prove this proposition, we present a simple lemma. 

Lemma 1. Let z = {zij} be any move for the univariate logistic regression. Then there 
exist ji < j2 (ind js < satisfying the following conditions. 

(a) zij^ > 0, zij^ < 0, zij.^ < 0, zij^ > ; 

(b) zij^ = 1 implies ji ^ ; 

(c) Zij^ = -1 implies J2 7^ Js / 

(d) zij = for ji <j< j2 and js <j< j^. 

Proof, (a), (b) and (c) are obvious from the constraint ([3]) and 2i+ = 0. We can assume 
without loss of generality that there exist ji < j2 such that zij-^ > 0, 2:1^2 < 0, > for 
1 < j < j2 and zij = for ji < j < 32- Since there exist ^2 < js < satisfying (a), (b) 
and (c), we can choose J3 and to satisfy (d). □ 
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We now give a proof of Proposition [2j 



Proof of Propsition [B We employ the distance reducing argument of iTakemura and Aoki 



2005 



Let X and y be two tables in the same fiber. Then z := x — y is & move. We 
can assume without loss of generality that there exist ji < j2 < J3 < j4 which satisfy the 
conditions of Lemma [1] and j2 — ji < j4 — js- Define js as js := j^ — (j2 — ji)- Then by 
applying a move 



we can reduce the Li distance between x and y, because at least one of the following 
operations can be performed to x or y: 



Jl 32 J5 J4 

+ 0+0+ + 



0+ 



0+ 0- 



Ji 32 35 3a 



0- 



0+ 



0+ 0- 



where 0+ denote that the cell frequency is nonnegative. 



0+ 0- 



□ 



Chen et al.l l2005l introduced a subset of B which still connects all fibers with X^j > 



0, Vj. IChen et al.l |2005l | did not give a proof of the following theorem. 
Theorem 1 fIChen et al.l |20o3)- 

The set of moves 
Bo = {zeB\j2= 31 + 1, 33 = J4 - 1} (6) 

connects every fiber satisfying (X+i, . . . > for the univariate logistic regression 

model. 

Proof. It suffices to show that any move in 2; G i3A(yi) of Proposition [2] can be replaced by 
a series of moves in Bq. To prove this it suffices to show that the Li norm of any move 
z G Ba{a), i-e., the Li distance between the positive part x = z~^ and the negative part 
y = z~ of z E B\(^A), is reduced by moves in Bq. Denote z := e^-^ — ej^ — ej^ + ej^. We can 
assume without loss of generality that ji < j2 < js < j4- We prove it by the induction on 
5 -=32 - jl = J4 -J3 > 2. 

When > or (x2ji+i, X2j4_i) > 0, we can apply z' := -ej^ + e^-^+i + 

^j4-l 



Cj^ to z and 



z + z' :-- 



'J 3 



-j4~l 



as seen from the picture below, where Zij = 0* denotes that Xij = yij > 0. 
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From the inductive assumption, we can reduce the Li norm of z + z' by moves in Bq. 
When X2j3+i) > or (a;2ji+i, Xij3+i) > 0, we can apply z' := -ej^ + e^^+i + 

— ej3_|.i to z and 

Z -\- Z . Cj-^j-i Gjgj^i + Bj^ 

as seen from the picture below. 
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Prom the inductive assumption, we can reduce the Li norm of z + z' by moves in Bq. In 
the case that (xij2_i, X2,j4-i) > or {x2j2-iy^iM-i) > 0' proof is similar. 
Suppose that 

X2Ji+l = Xij^-i = X2J3+I = Xij^-i = 0. 

We note that this implies that 

^l,jl + l > 0) ^2j2-l > O5 ^l,j3 + l > O5 ^2,j4-l > 0- 

Then there exists J3 < J5 < J4 such that 

xij, > 0, X2J5+1 > 

as in the following picture. 
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By applying 

z := —Cj^ + ej^+i + — e^^+i 
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and 



to z in this order, we obtain 

Z/ ~t~ Z ~l~ — 1 

as in the following picture. 
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From the inductive assumption, we can reduce the Li norm of z + z' by moves in Bq. 
In the case that Xij^+i = X2J2-1 = ^ijs+i = ^2^4-1 = 0, the proof is similar. □ 



3 Markov bases for models of Segre product type 

In the previous section we considered univariate Poisson regression and logistic regression. 
We now consider generalizing the results to multiple regression. In this section we show an 
explicit form of Markov basis for multiple Poisson regression. Therefore an extension of 
Proposition [1] to multiple regression is straightforward. In contrast, as we see in the next 
section, it is difficult to generalize the results of univariate logistic regression to multiple 
logistic regression. 
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Multiple Poisson regression is a Segre product of univ ariate Poisson regr essions. Quadratic 



Grobner bases of Segre products is already discussed in lAoki et al.l [20081 ] . However The- 
orem [2] below is about Markov bases (rather than Grobner bases) and it is applicable even 
if the component configurations do not possess quadratic Grobner bases. 

Consider two configurations A = (ai, . . . , aj) and B = (61, ... , b^), where aj and 
are column vectors. We assume the homogeneity, i.e., there exist weight vectors w,v such 
that {w, ttj) = 1, Vj, {v, bfc) = 1, Wk. The configuration A ® _B of the Segre product of A 
and B is defined as 

A(S)B = (^a^®bk, J = l,...,J,k = l,...,K^, aj ® bk = (j^^ . 

If both A and B are configurations of the form ([T]) for the univariate Poisson regression 
model, then A ^ B corresponds to the bivariate Poisson regression model, where Xj^ 
is independently distributed according to Poisson distribution with mean ^jk, which is 
modeled as 

log fijk = ^ + aj + Pk, j = 1, . . . ,J, k = 1, . . . ,K. 

Let X = {X jk) j=i,...,j,k=i,...,K denote a table of observed frequencies. The sufficient 
statistic for the Segre product A® B \s given by 

Therefore z = [zjk) is a move for A® B ii and only if 

= ^ ajZj+, = X] bkZ+k- (7) 

j k 

Given Markov bases Ba and Bb for A and B, respectively, our goal is to construct 
a Markov basis for the Segre product A <® B. Denote the elements of Ba by = 
(zf, . . . , zf). Let zf'^ = max{zj^, 0) be the positive part and zf'~ = max(— zj^, 0) be the 
negative part of z^. Let degz"^ = ^^=1 = ^^=1 be the degree of z^. Now is 
uniquely written as 



deg 2 



E 

h=l 

where ji < ■ ■ ■ < jdegz^ and j[ < ■ ■ ■ < Jdeg^-*- denote a J x K table with 1 at the 

cell (j, k) and everywhere else. Now choose arbitrary 1 < ki, . . . , k^^gz^ ^ K and define 

deg 2:-'* 

Z^{ki, . . . , fcdeg^^) = Yl ^^it^^i^ ~ ^fh^h)- 

h=l 

We call . . . , k^egz^) a "distribution" of z^ by coordinates ki, . . . , k^^gz^- Note that 

ki, . . . , kdcgz^ are not ordered. Similarly define the distribution . . . , jdeg^s) of a 

move z^ G Bb- 
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In addition to these moves we also consider the basic moves z{ji,j2] ki, /C2) = ^jik^ + 
e^jfea ~ ^jik2 ~ ^j2ki of the form ^ j . We now have the following theorem. 

Theorem 2. The set of basic moves and the set of moves of the form z^{ki, . . . , /Cdegz^)? 

1 < ki,. . . .k^e^^A < K, Z"^ e Ba, Z^ijl,---JdegzB), 1 < jl , • • • , Jdcg < J, E Bb, 

form a Markov basis for the Segre product B. 

A proof of this theorem is given in Appendix. In Theorem [2] we have considered Segre 
product of two configurations. By a recursive argument, a Markov basis for the Segre 
product of arbitrary number of configurations Ai(E) ■ ■ ■ <^ is given as follows. Let Baj 
be a Markov basis for the configuration Aj, j = 1, . . . ,m. Write [J] = {1, . . . , J} and let 

J. = [J^]x ■■■ X [Jj_i] X [Jj+i] X ■ ■ ■ X [J„,]. 

Let z"^^ G Ba and let ki, . . . ,k, a, G jf,-. Now define 

^3 ^ ' ' dcg Z 3 ^ J 

dog z'^i 
h=l 

where e^-^fc is an m-way table with 1 at the cell (j, k) and everywhere else. Then we 
have the following corollary to Theorem [2l 

Corollary 1. The set of square- free degree two moves for the complete independence model 
of Ji X ■ ■ ■ X Jm contingency tables and the set of moves of the form z^^{ki, . . . , fcdcgz-*); 
fci, . . . , fcdcgz-^ G Sj, j = 1, . . . ,m, form a Markov basis for the Segre product Ai®- ■ -^Am- 

Minimality of the Markov basis constructed in Theorem [2] is not clear at the present. 
However the maximum degree of moves in the Markov basis for Ai ® ■ ■ ■ ® Am is bounded 
by the maximum degree of moves in B^i , ■ ■ ■ Ba^ ■ 

4 Connectivity of fibers of positive marginals in bi- 
variate logistic regression 

In this section we consider the extension of the results in univariate logistic regression 
model to bivariate logistic regression model. Let {1, . . . , J} and {!,..., K} be the sets 
levels of two covariates. Let Xijk and X2jk, j = ^, ■ ■ ■ , J, k = 1, . . . ,K, be the numbers 
of successes and failures, respectively, for level {j,k). The probability for success pijk is 
modeled as 

logit(pi,fc) = log =^ + aj+ Pk, (8) 

j = 1, . . . , J, k=l,...,K. 
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The sufficient statistics for this model is X]^=ii^ii+' X^^i ^^^i+fc' ^ 



+jk; 



Vj, k. 



Hence moves Z = {zij^) for the model satisfy 



K 

it=i 



0, z+jk = 0, Vj, k. 



Let 



5 



1 



and let A be defined as in Then the configuration for the bivariate logistic regression 
model is the Lawrence lifting of Segre product A{A^B). Here we consider a set of moves 
which connects every fiber satisfying X^jk > 0, Vj, k. 

Definition 1. Let ejk = {eijk) be redefined as an integer array with 1 at the cell (Ijk), 
— 1 at the cell (2jk) and everywhere else. Define Ba(a^b) the set of moves z = (zijk) 
satisfying the following conditions, 



1. Z f^jiki ^^2^2 

Uuk^) - {j2,k2) 



-- (j3, ^3) - (j4, ^4) 



BA{A(g,B) is an extension of Ba{A) in Proposition [2] to the bivariate model ([8]). We note 
that the (z = l)-slice of a moves (z^k) in Ba{A(SiB) is a move of the Markov basis defined 
in Theorem [2l Now we present the main theorem of this paper. 

Theorem 3. BA(Aig)B) connects every fiber satisfying X^j^ > 0, Vj, k. 

A proof of this theorem is given in Appendix. We give some examples of moves in 

BA{Aig)B) ■ 



(1) k. 



kd 





Jl J2 J3 


Ji 


ki 


1 -1 -1 


1 


(4) ki = k2 and j2 


= J3 




jl J2 j4 




ki 


1 -1 




k3 


-1 1 





(2) k, = ■ 




= ki and js = ja 


(3) fci = 


k2 {ks = 


ki 


) 




Jl 


J2 


J4 






Jl 


J2 j3 




ji 






ki 
k3 


1 



-1 
-1 





1 


ki 


1 


-2 


1 


















(5) ij2,k2 


) = 


(j3, ^3) 


(6) k^ = 


ki and j2 




--j3 




jl 




ji 






jl 


J2 j4 






ki 


1 










k2 





-1 






k2 





-2 







ki 


1 


1 






ki 








1 




k3 





-1 







5 Numerical examples 

5.1 Data on coronary heart disease incidence 



Table T] refers t o coro nary heart disease incidence in Framingham, Massachusetts Cornfield 



1962 



Agrestil . Il990| . A sample of male residents, aged 40 through 50, were classified on 
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Table 1: Data on coronary heart disease incidence 



Serum Cholesterol (mg/lOOml) 



Blood 
Pressure 


1 


2 


3 


4 


5 


6 


7 


< 200 200-209 


210-219 


220-244 


245-259 


260-284 


> 284 


1 < 117 


2/53 


0/21 


0/15 


0/20 


0/14 


1/22 


0/11 


2 117-126 


0/66 


2/27 


1/25 


8/69 


0/24 


5/22 


1/19 


3 127-136 


2/59 


0/34 


2/21 


2/83 


0/33 


2/26 


4/28 


4 137-146 


1/65 


0/19 


0/26 


6/81 


3/23 


2/34 


4/23 


5 147-156 


2/37 


0/16 


0/6 


3/29 


2/19 


4/16 


1/16 


6 157-166 


1/13 


0/10 


0/11 


1/15 


0/11 


2/13 


4/12 


7 167-186 


3/21 


0/5 


0/11 


2/27 


2/5 


6/16 


3/14 


8 > 


186 




0/1 


3/6 


1/10 


1/7 


1/7 


1/7 


Source : 


Cornfieldl 1962] 



blood pressure and serum cholesterol concentration. 2/53 in the (1,1) cell means that 
there are 53 cases, of whom 2 exhibited heart disease. We examine the goodness-of-fit 
of the model (El) with J = 7 and K = 8. We first test the null hypotheses Hq, : a = 
and B.j3 : P = versus (IHl) using the likelihood ratio statistics La and Then we have 
La = 18.09 and Lp = 22.56 and the asymptotic p- values are 2. 107 x 10"^ and 2.037 x 10"^ 
respectively, from the asymptotic distribution xl- We computed the exact distribution 
of La and Lp via Monte Carlo Markov chain (MCMC) with the sets of moves Br{A) and 
Bq discussed in Section [21 See the last paragraph of Section El on sampling under Ha 
and Hp. In all experiments in this paper, we sampled 100,000 tables after 50,000 burn-in 
steps. Figure [H and [2l represent histograms of La and Lp. The solid lines in the figures 
represent the density function of the asymptotic distribution xl- The estimated p- values 
are 1.0 x 10~^ for all cases. Therefore both Hq, and H/3 are rejected. We can see from the 
figures that there are little differences between two histograms computed with Br{A) and 
Bo- 

Next we set (jSj) as a null hypothesis and test it versus the following ANOVA type logit 
model. 

Hi : logit(pijfc) = log ( ^^-^^ ) = fi + aj + pk, (9) 

Vi -PijkJ 

where ^^=1 a-,- = and J2k=iPk = by likelihood ratio statistic Lq. The value of Lq is 
13.07587 and the asymptotic p- value is 0.2884 from the asymptotic distribution Xu- 

We 

computed the exact distribution of Lq via MCMC with Br(A(g)B) defined in Definition [H 
As an extension of Bq in Theorem [H to the bivariate model (IE]), we define Bq by the set 
of moves z = ej^ki - ej^ki - ej-^ki + fij^ki satisfying (ji, ki) - (j2, h) = (js, h) - (j4, k^) is 
either of (±1, 0), (0, ±1), (±1, ±1) or (±1, =i=l). We also computed the exact distribution 
of Lq with Bq. Figure [31 represents histograms of Lq computed with Br(A»B) and Bq. The 
estimated p-values are 0.2706 with -Br(A®_B) and 0.2958 with Bq. Therefore the model ([8|) 
is accepted. 

The p-values estimated with B^(A(^$B) and B^ are close and there are little differences 
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02468 10 02468 10 



(a) A histogram with Br(A) (b) A histogram with Bq 
Figure 1: Histograms of La via MCMC with Br(A) and Bo 




02468 10 02468 10 



(a) A histogram with Br(A) (b) A histogram with Bq 
Figure 2: Histograms of via MCMC with Br(A) and Bq 




(a) A histogram with Ba{A(^b) (b) A histogram with Bq 
Figure 3: Histograms of Lq via MCMC with Ba(A(S)B) and B^ 

between two histograms. From the results of Theorem[T]and this numerical experiment, Bq 
is also expected to connect every fiber with positive response variable marginals. However 
the theoretical proof of it is not clear at the present and is left to our future research. 
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5.2 Data on occurrence of esophageal cancer 



Table 2: Data on occurrence of esophageal cancer 













Alcohol 1 2 


3 


4 


5 


6 


Consumption 25-34 35-44 


45-54 


55-64 


65-74 


75+ 


Low 0/106 5/169 

1 High 1/10 4/30 


21/159 
25/54 


34/173 
42/69 


36/124 
19/37 


8/39 
5/5 


Source : Breslow and Dav [1980] 



The second exampl e is from Table 4.16 in IChristensenI 1997l | (data source is from 
Breslow and Day 1980l |). Table [2] refers to the occurrence of esophageal cancer in French- 
men which were classified on ages and dummy variable on alcohol consumption. We test 
the goodness-of-fit of the model ([8]) with J = 6 and = 2 by likelihood ratio statistics Lq 
via MCMC Then the value of Lq is 20.89 and the asymptotic p-value is 0.0003330 from 
the asymptotic distribution xl- We computed the exact distribution of Lq via MCMC 
with Br(A(i,B) and B^. Figure H] represents the histograms of Lq. The estimated p-values 



are 0.00011 with Br(A(^B) 
significance level of 1%. 



and 0.00055 with Bq. Therefore the model ([8]) is rejected at the 



(a) a histogram with B\(^ais,b) (b) a histogram with B^ 

and Bn 



Figure 4: Histograms of Lq via MCMC with B 



6 Concluding remarks 

In Theorem [3] we showed the connectivity result for bivariate logistic regression. A natural 
extension of Theorem [3]to m covariates is given as follows. Let j = (ji, . . . ,jm) denote 
the combination of m levels and let ej denote an array with 1 at the cell (1, j) and —1 at 
the cell (2, j). Define B\(Ai<s,---^Am) as the set of the following moves z: 

1. z = ej^ — — 6^3, +6^^ 
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2- ji- 32 = js - ji 



Then we conjecture the following. 

Conjecture 1. The set of moves i3A(Ai0---(g)Am) connects every fiber with positive response 
marginals for the logistic regression with m covariates. 

The separation lemma and some steps of the proof of Theorem [3] can be easily gener- 
alized to multiple logistic regression. However many steps of our proof, especially those 
for Cases 3 and 5, are restricted to the two-dimensional case. 

As discussed in Section [5] it seems that we can further restrict to the set of moves 
z = — — -|- Bj^, where the elements of ji — j2 = js — j4 are ±1 or 0. Hence a 
stronger conjecture (even for the case of m = 2) is given as follows. 

Conjecture 2. The subset of moves from B^A-^^^...^Am) such that the elements o/ji— = 
J3 " ji fl^e ±1 or connects every fiber with positive response marginals for the logistic 
regression with m covariates. 

In this paper we considered logistic regression, which is the Lawrence lifting of Pois- 
son regression. Our Theorem [2] describes Markov bases for a general Segre product of 
configurations. Therefore it is interesting, in practice, to investigate connectivity result 
for Lawrence lifting of a general Segre product of configurations. 

In the bivariate logistic regression, it is interesting to test the null hypothesis that 
the coefficient of one of the covariates is zero. Generating random samples under the 
null hypothesis is simple because it reduces to univariate logistic regression as follows. 
In ([8]) consider the null hypothesis Hq : P = 0. Given observed data (xijk), we can 
generate random sample from the null conditional distribution by MCMC procedure for 
the marginals j = 1, . . . , J. Then for each j, we can sample Xijk, k = 1, . . . ,K, 

by the random sampling without replacement. 



Appendix 

A Proof of Theorem [2 

Let X and y be two tables in the same fiber. Write z = x — y. First consider the case that 
X and y already have the same marginals: 

= Zj+,yj, = z+k,yk. 

Then, as is well known for two-way complete independence model, we can use the basic 
moves to move from X to Y. Note that ([7]) is always satisfied in these steps. 
Next consider the case that the row sums are already the same 

Zj+ = 0, j = 1, . . . , J, 
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but the column sums are not yet the same. For the moment, ignoring joint frequencies, 
just look at the column sums of x and y: 

{x+i, . . . ,x+k), {y+i,...,y+K) 

We can use the moves of Bb to move from the marginal frequency (x+i, . . . , x^k) to the 
marginal frequency (y+i, . . . , y+x)- However, of course we have to worry about the joint 
frequencies and the row sums. Here the idea is that we can "distribute" moves of iS^ to 
the cells of the J x K table, in such a way that we never disturb the row sums. This way, 
we can make column sums equal, while always keeping the row sums equal. Consider a 
situation that a move of Bb can be added to (x+i, . . . , x^k)- Then we have 

x+k — ^fc ' ) k = 1, . . . , K. 

This shows that in each column k with Ck = Zf,' > 0, there are at least Ck positive 
frequencies of x, i.e., there exists indices ji^k, ■ ■ ■ ,jck,k such that 

(xifc, . . . , xjk) - {ej^^, + ■■■ + ej^^J > 0. 

Here "> 0" means that every component of the left-hand side is non-negative. Collect 
the indices . . . ,jck,k for all k with z^~ > as ji, . . . , Jdeg^s- Then z^{ji, . . . , Jdcg^s) 
can be added to x. Note that z^ is added to the marginal frequencies (x+i, . . . ,x+k), 
but the move does not change the row sums of x. This argument implies that the set of 
moves z^{ji, . . . , jdeg^s) are sufficient for connecting two tables with the same row sums. 

Lastly we consider the case that neither the row sums nor the column sums are the 
same for x and y. We can employ a "greedy algorithm", in which we first look at the 
row sums only and try to make the row sums equal, because the column sums can be 
adjusted later by the above argument. Now in the above argument, with the roles of the 
rows and the columns interchanged, we can ignore the fact that the column sums are not 
yet equal. We can use the same procedure as above. Therefore, by applying a move of 
the form z^{ki, . . . , ^degz-*) we do not change the column sums of x and y. Then we can 
make the row sums of x and y equal, while not changing the column sums of x and y. 

B A separation lemma 

Here we prove a lemma, which is needed for our proof of Theorem 121 

Lemma 2. Let I = [J] x [K] and let and S~ be disjoint subsets of I satisfying the 
following properties: 

1. (j, k) e S+, f <j, k'<k ^ if, k') G S+. 

2. (j. A;) e 5", 3' >j, k'>k ^ (j', k') e S~. 

3. There are no distinct four points (ji, ki) G S*"*", (j2, ^2) ^ S", (ja, k^) ^ S^, (j4, k^) G 

and there are no distinct four points {ji,ki) G 5'~,(j2,^2) ^ •S''^, (js, /i^s) ^ 
S~ , (j4, k4^) G such that 

(ji, ^1) - (j2, k2) = (j3, k^) - (j4, ki). 
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Then there exists a line with rational slope separating S'^ and S , i.e. there exist integers 
a,b,c, {{a,b) ^ (0,0)), such that 

5+ C {(j,A;) G J| aj + 6A; <c}, 5" C {(j, A;) G J | aj + 6A; > c}. (10) 

Proof. The lemma obviously holds if or S~ is empty. Therefore we only need to 
consider case that 5"^ and 5"" are non-empty. Define ji = min{j | 3{j, k) G S~} and for 

j ^ I?/, ?/ + 1 Jj defi ne /(j) = mm{k \ {j,k) G S~}. Let /* be the largest convex 

minorant [Morigutil . Il953l | of /(j), j G {ji,ji + 1, . . . , J}, i.e. /*(■) is the maximum among 
convex functions not exceeding /(j) for each j G {ji,ji + 1, . . . , J}. Then /* consists of 
finite number of line segments. Let ji < be endpoints of a line segment and let L*^ 
denote the line segment. Then (ji, /(ji)), (j4, fij^)) G S~. Also by construction of /*, 

U,k)eS-, j^<j<u k>nj). 

Therefore every point strictly below L*^ belongs to S^. Consider the rectangular region 
of integer points 

= {Ji: • • • : J4} X {/(J4), • • • , /(ji)}- 

If there exists a point (j2,^2) E S'^ H Rj^^j^ strictly above the line segment Lj'^^^, let 
(j3! ^3) = (jij^i) ~ {32, k2) + (j4, A;4) G (5'+)'" n -Rjij4 and condition 3 of the lemma is 
violated. This shows that no point of Rj^^j^ strictly above L^^^-^ belongs to S~ . Also the 
points above Rj^j^ belong to by the monotonicity condition (2). Therefore -Z^j^j^ is a 
separating line for the interval {ji, . . . , ^4}. 

Now we similarly construct the smallest concave majorant /*(j) for 5*". Then by a 
hyperplane separation theorem for two convex sets, there exists a line with rational slope 
between /*(j) and /*(j). This prove the lemma. □ 



C Proof of Theorem [3 

Let X := {xijk} and y := {Hijk} be two 2 x J x K tables in the same fiber satisfying 
x+jk = y+jk > 0. Then z := {zijk} = x — y is a. move for A{A (g) B). Let denote the 
(z = l)-slice of z. As mentioned in Section HI z satisfies z+jk = 0, — 0, yi,j, k, and 

J K 

J2jzi,+ = 0, ^/czi+fc = 0. (11) 

j=i k=i 

Note that Zi++ = implies 

J2jzih = ^ 5^(J-j + l)^y+ = 0. (12) 

Similarly J2k=i kzi+k = -v^ J2k=i(^ — k + l)zi+fc = 0. This implies that when we 
consider a sign pattern of a move, we can arbitrarily choose directions for two factors j 
and k. 
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Let X+ and X be the multisets of indices defined by 

J+ := {(j, k) I z,,k > 0}, I- := {(j, k) I zyfc < 0}, 

where the multiphcity of (j, k) in X+ and X~ is |2:ij/fc|. 

Suppose that (ji, ki) G X+, (j2, k2) G X" and ji < j2- Then we note that there exist 
is < J4! ^3 and k^ satisfying 

(j3, k-s) G X- \ {(j2, k2)}, (j4, A;4) G X+ \ {(ji, A;i)} (13) 

from (fTTl) . If /ci < and /cs > /C4, there exists k^ < /cg, js and jg satisfying 

(J5, A;5) G X- \ {(j2, A;2), (j3, /cs)}, (je, /cg) G X+ \ {(ji, /ci), (ja, k^)}. 

Write 72; ^2) = ^jiki—e-j^k2 ■ When a move 2; includes ?/(ii,i2;ji,j2), we denote 
it by y{ii,i2] 31^32) C z. 

Case 1. We first consider the case where there exist jo? ji-, J2, ^o? ^1 and /c2 such that 

^liofci > 0' ^iiofe2 < (14) 
^lii fco 

> 0, zij^k^ < 0. (15) 

Without loss of generality we assume ji < ^2 and ki < k2- Let 5*+ = {(j, A;) | 3(j', k') G 
J < 3',k < k'}. Similarly define S~ = {{j,k) \ 3{f,k') G X", j > f,k > k'}. We 
only need to consider the case that the condition 3 of Lemma [2] is satisfied. Also, if 
or is not monotone in the sense of conditions 1 and 2 of Lemma [2], we can reduce the 
Li distance between x and y. This can be seen as follows. If or S~ is not monotone, 
then we can find a pattern in Figure [5] (or a vertical pattern of this). Without loss of 
generality let j2 —ji < ji — jz and define js := j^ — (j2 — ji)- For simplicity assume j2 < js 
or ko ^ k^. Then by applying 

to 2;, we can reduce the Li distance between x and y by at least four. The case of ko = ki 

3i 32 h k H 



A:o + 0---0- ki - Q ■■■ + 



Figure 5: Case 1 or Case 2-1 

and j2 = is needs a special consideration, but the monotonicity holds with respect to the 
horizontal separating line through (/co,j2)- Therefore it suffices to consider the case that 
and S~ are monotone in the sense of conditions 1 and 2 of Lemma O Then 

^(aj + hk + c)zijk = 

j,k 
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implies that non-zero elements zijk 7^ only exist on the line {(j, k) \ aj + bk + c}. Then 
the problem reduces to the univariate logistic regression. 

Case 2. Next we consider the case that only one of the patterns of f|T^ or f|T5l) exists. 
Without loss of generality, we assume that f|T5|l holds and from Lemma [1] we assume that 
there exist ji < j2 < js < Ja, such that 

In this case either a pattern of signs in Figure E] or a pattern in Figure M has to exist in 



ki + 



J3 



32 







J4 



Figure 6: Case 2-2 



Case 2-1. The case of Figure [51 

In this case we can reduce the Li distance between x and y as in Case 1. 
Case 2-2. The case of Figure [61 

In the case of Figure [6l we distinguish two subcases depending on j2 — Ji < J4 — J3 or 
32 -ji > 3i- h- 

Case 2-2-1. js - ji < Js - 3^- 

Let js := is + (j2 - ji)- By applying 

to 2, we reduce the Li distance by four. 

Case 2-2-2. - ji > ji - Js- 

In this case we prove the theorem by induction on — j^. When — ja = 0, the 
problem is reduced to Case 1. Therefore we assume that — js > 0. 

Case 2-2-2-1. (a^ij^+i^fe,, Xij^-i^^J > or {x2,j^+i,k^, X2,j^-i,k4) > 0. 
In this case we can apply 
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to z and then 

V{k-,H - ^;h,h) C z + Z2b, \\z + Z2b\\i = \\z\\i. 

Hence ||2;||i is can be reduced by moves of B^AigiB) from the inductive assumption. 

In the case where {xij2-i^k2y ^ijs+im) > or (x2j2-i,fe2 5 ^zjs+i.fca) > 0, the proof is 
similar. 

Case 2-2-2-2. {xij^+i^ki,X2,js+i,k3) > or {x2j^+i,ki,Xij^+i,k3) > 0, 
In this case we can apply 

to z and then 

+ 1, ji] h, h) C Z + 22c- 

Therefore \\z\\i can be reduced by moves of Ba(AiS)B) from the inductive assumption. 

In the case where (xij2_i,fc2, X2j4_i,fe4) > or {x2j2-i,k2^^iji-iM) > 0) the proof is 
similar. 

Case 2-2-2-3. (xij^+i^^j, X2j2-i,fc2) ^ijs+i.fes' ^2^4-1,^4) =0. 
In this case we have 

(^2ji+l,A:u 2;ij2-l,fc2) ^2,j3+l,A:3; 2;ij4_i^fc4) > 0. 

Then there exists ji < js < j2 such that {x2j^ki,Xij^+i^ki) > 0. Hence we can apply 

^2d ■~ ~^jiki + + ^jski ~ ^is+l.fci 

and 

2 

to z in this order and then we have 

lk + ^2d + ^2dlli = Iklli and ?/(ji,A;i + 1; j2,/c2) C z + 4d + 22d- 
Hence theorem holds from the inductive assumption. 

In the case where (x2ji+i,fci, Xij2-i,fc2? ^2j3+i,fc35 a;ij4_i_fc4) > 0, the proof is similar. 

Case 3. We now consider the case that there exist no jo? ^ii ^2 satisfying fll4l) and there 
exist no /co,Ji,j2 satisfying f|T5|) . From Lemma [H either of the patterns of signs as in 
Figure [7] and Figure [8] has to exist in z^. Here we consider the case that patterns in 
Figure [7] exist. The case of Figure [8] will be treated as Case 5 below. We make various 
subcases depending on the sizes of two rectangles in Figure [71 
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+ 







J3 







Jl 



ki 







J2 



Figure 7: Case 3 



A;2 

32 h 



J3 



Figure 8: Case 5 







J4 



J4 



Case 3-1. ja - ji > J4 - js and k2-ki>h- k^. 

In this case the left rectangle contains the right rectangle in Figure [71 Define (js, k^) 
by 

(is, ^s) := (jl, ki) - (js, /cg) + (j4, A;4). 

Then 

reduces the Li distance by four. 

In the case where ja — ji < ji — js and k2 — ki < — k^, the proof is similar. 

Case 3-2. When there is no inclusion relation between two rectangles of Figure U\ it 
suffices to consider the case of ja — ji > 3a — js and /c2 — /ci < k^ — k^. We prove the 
theorem by induction on / := (/c2 — ki) + (^4 — j^). If / = 0, the theorem holds from 
Case 1. 

Case 3-2-1. {xij^^k^+i, xij^^k^^i) > 0. 
In this case we can apply 

and then we have 

yiju32; kl + I,k2) C Z + Z3b, 2/(j3, J4; ^4, k4-l) C Z + Zsb. 

From the inductive assumption the theorem holds in this case. 
Also in the following cases, the proof is similar. 



20 



• {Xi,ji+l,kn Xij^-i^ki) > ; 

Case 3-2-2. {xij^^ki+i,X2j3,k3+i) > 0- 
In this case we can apply 

and then we have 

y{ji,j2; h + I,k2) C z + zsc, yijsji] h + l,ki) C z + Z3c- 

From the inductive assumption the theorem holds in this case. 
Also in the following cases, the proof is in the similar way. 

• (2;2,ji,fei+i; 3;ij3^fc3+i) > ; 

• (-^jji+lifcu •^1*^3+1,^3) ^ , 

• i.'^i,j2 — ^,k2i-^i*,j4—l,ki) ^ 0, 

where i* := 3 — i. 

Case 3-2-3. X2j^,ki+i = 2^1^2,^2-1 — X2j3,k3+i ~ ^ijiM-^ ~ ^■ 

From the result of Case 3-2-1 and Case 3-2-2, it suffices to consider the case where 

2^2ji,fci+i = 3;ij2,fc2-i — 3^273, fc3+i = a^ii4,fc4-i = 0. (16) 
We note that (fT6|) implies 

(^^lil.fcl + l; 3;2j2,fc2-l5 ^Ij3,fc3 + l) ^2j4,fc4-l) > 0- 

Therefore there exist ji < J5 < j2 and ki < < k2 satisfying either 

xij.ks > X2j,,k5+i > (17) 

or 

xij.k, > X2,j,+i,ks > 0. (18) 

Case 3-2-3-1. Xij^k^ > and X2j^,k5+i > f|T7l) . 
In this case we can apply 

^3d ■~ ~^jiki + ^ji,fci+l + ^jsfcs ~ ^J5,fc5+1 
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and 

2 

to z in this order. Then we have \\Z + + -23^||i = ||2||i and 

y{hd2] ki + l,k2) d z + zIj^ + zla, y{j3, ji] ks, k^-l) C z + zla + zl^. 
Hence from the inductive assumption the Li distance can be reduced by moves in 

^K{A®B) ■ 

Case 3-2-3-2. xij^fc^ > and 0:2^5+1,^5 > f|T8l) . 

In this case we further consider subcases depending on the value of Xij^+i^fc. 

Case 3-2-3-2-1. > 0. 

From the result of Case 3-2-1 and and Case 3-2-2, it suffices to consider the case 
where 

2^2,ji+l,A:i = ^IJi-lM ~ ^2,j3+l,fc3 = Xl,j2-IM = 0- (19) 

We note that (IT^ implies that 

(•^Ijl + lifcl' •^2j'4-l,fc45 •^I,i3 + l,fc37 2:2^2-1,^2) ^ 0- 

Since (fTSi) is satisfied, we can apply 

4e ■= -^hki + eji+i,fci + ej5fc5 - ej5+i,fc5 

and 

2 

^3e ~^j5k5 "I" ^js+lM ^J4-l,fc4 ~ ^J4A;4 

in this order. Then we have \\z + z^^ + z^J\i = \\z\\i and 

yUi + 1, J2; A;i, A;2) C z + zl^ + zl^, y{js, J4 - 1; k^, k^) C z + z\^ + zl^. 
Hence from the inductive assumption, distance can be reduced by moves in 

If any oi x^^j^-x^k^-.^x.h+iMi^'^.u-^M positive, the same argument can be applied. 

Case 3-2-3-2-2. (Case 4) Xij^+i,^^ = X2,j2-i,fe2 = ^1,^3+1,^3 = 2:2^4-1,^4 = 0. 
For readability, we relabel this Case 4. In this case 

(2:2 Ji + l,fci, a;ij2-l,fc2 5 2:2 J3 + I,fc3, 3^1^4-1,^4) > 0- 

Then there exists ji < je < 3i and k\ <k^ < k2 satisfying either 

X2Kk(, > 0, a^ijg+i.fcs > (20) 

or 

X2j6k6 > 0, Xijg,fcg+i > 0. (21) 
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Case 4-1. The case that fl20|) is satisfied. 

In this case the proof is in similar to Case 3-2-3-1. 



Case 4-2. The case that fl20|) is not satisfied. 
In this case 



Xljk-, =0, j = ji + 1, . . . , j2, X2j3k = 0, k = h + 1, . . . ,k4. 



(22) 



We can assume without loss of generality that j2 < js- Then we note that 



(i?, kj) ■■= (j4, h) - (j2, h) + (ji, fci) e J 



where J := [J] x [A']. 



Case 4-2-1. X2j^k7 > or ^/i^^fcy > 0. 
In this case we can apply 




to z and we can reduce the Li distance by four. 

Case 4-2-2. X2j7fc7 = and yij^fc^ = 0. 
In this case we have zij^kr > 0. 

Case 4-2-2-1. The case that there exists jj < js < js such that Zij^kr < 0. 
In this case we can prove the theorem in the same way as Case 2-2. 

Case 4-2-2-2. The case that Zijk^j > for all J7 < j < js. 

From the condition fl22|) there exists jg satisfying either of the following conditions, 

(i) j? < jg < ja, 2iigfc7 > and Xij^+i^^^ > ; 

(ii) j? < jg < js) ^ijskr = ^iig+i,fe7 = 0, X2jQk7 > and Xij^^i^ky > 0. 

Case 4-2-2-2-1. The case that (i) is satisfied. 
In this case by applying the move 



we have \\z + Z4b\\i 



z\\i and 



y{ji,j2; h, k2) c z + Zib, yUsJi - i; ^3, ki) c z + z^b- 



Hence the theorem holds from the inductive assumption. 
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Case 4-2-2-2-2. The case that (ii) is satisfied. 
In this case by applying the move z^f, and 

in this order and then we have \\z + z^b + z^dli = \\z\\i and 

y{ji + l,j2] h, k2) C z + Z4b + Z4c, y{h, J4 - 1; ki) ^z + z^b + z^c 

Hence the theorem holds from the inductive assumption. 

Case 5. We now consider the case where z^ contains patterns of signs in Figure [H] and 
does not contain patterns of signs in Figure [71 We show that if zi contains the pattern of 
signs in Figure [9l we can reduce the Li norm z or otherwise z is not a move. The proof 
is by induction on 

/ := min ((js - ji) + - /ci), {ji - js) + (^3 - k4)) ■ 
When / = 1, theorem holds by Case 1. 
Case 5-1. /c2 > k^ 

Based on the argument in Case 3-2-1 and 3-2-2, we only need to consider the case 
that z^ contains patterns in Figure where zij^ = 0* and zijk = 0* denote Xijk = 
yijk > and X2jk = y2jk > 0, respectively. Define two set of cells Ai and A2 as in 
Figure [H Then there exist js, k^, Jq and /cg such that 

Ji < is < j2, k < k < i4, ki<k^< k2, ki<kQ< k^, 
as represented in Figure [9](i). Then we can apply the move 

Z5b ■ ^jakQ 6^6,^6+1 ^ ^i4fc4 ^i4,fc4+l 

to z in this order and z' := z + z^a + z^b is expressed as in Figure [9]^ii). Suppose that 
there exists (j, k) G Ai such that zijk < 0. Then zijk^ = and hence there exists 
k < k' < k4 such that 

zijk' < 0, Zij^k'+i = 0. 
Therefore we can apply the move 

Z5c '■= —^jsk^ + ^is.fcs+l + ^jk' — ^j,k'+l 

and z" := z' + z^^ satisfies 

Ik'lli = Iklli. yUi^k^ki^h) c z". 
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Figure 10: Case 5-2 



Therefore the theorem holds by the inductive assumption. 

Similarly we can prove the theorem in the case where there exists (j, k) G A2 such 
that zijk > 0. 

Now we suppose that Zijk > for {j,k) G Ai and z^k < for {j,k) G Ai. Since 
there does not exist the pattern in Figure [TJ there exist k^ < k-j < k^ such that 
Zijk > for k < k-i and ^ij^ < for /c > /C7. This contradicts the condition 
Y2k=i ^^ijk = and hence z is not a move. 

Case 5-2. A;2 < k^ 

By using the same argument, we only need to consider the case that contains 
patterns in Figure [TUT i). Then both and S~ is monotone in the sense of condi- 
tions 1 and 2 of Lemma [21 Therefore if 2; is a move, we can reduce Li norm of z 
from Lemma [2] in the similar way to Case 5-1. 

Acknowledgments. We are grateful to Kazuo Murota for valuable comments on the 
separation lemma and to Ian Dinwoodie for constructive suggestion on the numerical 
examples. 
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